Information processing system and method for controlling the same

ABSTRACT

The present invention concerns one of the plurality of first storage apparatuses, prior to a file migration to the second storage apparatus, notifies to the second storage apparatus of file migration information being information relating to the file migration, the second storage apparatus calculates an increment of a load on the second storage apparatus that is generated by the file migration based on information written in the file migration information, the second storage apparatus determines whether the file migration is allowable based on a current load on the second storage apparatus itself and the increment, the second storage apparatus notifies the determination result to the one of the plurality of first storage apparatuses that has notified the file migration information, and the one of the plurality of first storage apparatuses determines whether to migrate the file to the second storage apparatus based on the determination result.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a Continuation of U.S. application Ser. No.13/201,091 (National Stage of PCT/JP2011/004133), filed Aug. 11, 2011,incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to an information processing system and amethod for controlling the information processing system.

BACKGROUND ART

Recently, there has been developed a technology related to aninformation processing system in which storage systems are respectivelystructured in bases and the like of a company and files respectivelygenerated in the storage systems of the bases are aggregated in a datacenter and stored therein.

PTL 1 discloses a technology that, in an information system including afirst storage site and a second storage site which is located at aremote site, the first storage site includes a first storage systemwhich provides a higher hierarchy of the storage and a second storagesystem which provides a lower hierarchy of the storage, the secondstorage site includes third and fourth storage systems, whichrespectively receive remote replication from the first and secondstorage systems and, when a file is migrated, a stubbed file whichdisplays the storage location of the migrated file in the second storagesystem is saved at a position of the migration source file while, whenthe copies of the migrated file and the stubbed file are replicated tothe fourth and third storage systems respectively, information forlinking the copy of the stubbed file in the third storage system to thecopy of the migrated file in the fourth storage system is transmitted tothe second storage site.

CITATION LIST Patent Literature

[PTL 1]

Japanese Patent Application Laid-Open Publication No. 2009-289252

SUMMARY OF INVENTION Technical Problem

In such a case, the load on the storage system of the data centerincreases along with such as the increase in the number of bases and theamount of data of the handled files, when the scale of the informationprocessing system increases.

Therefore, there is a demand for technology for appropriatelycontrolling the load on the data center and improving the reliability ofthe entire information processing system.

The present invention has been made in view of the above-mentionedbackground and a purposes thereof is to improve the reliability of theinformation processing system.

Solution to Problem

One aspect of the present invention for achieving the above-mentionedpurpose is an information processing system comprising: a plurality offirst storage apparatuses that receive data I/O requests in file unitssent from an information processing apparatus, and perform data I/O; anda second storage apparatus that is communicatively coupled to each ofthe plurality of first storage apparatuses, wherein one of the pluralityof first storage apparatuses, prior to a file migration to the secondstorage apparatus, notifies to the second storage apparatus of filemigration information being information relating to the file migration,the second storage apparatus calculates an increment of a load on thesecond storage apparatus that is generated by the file migration basedon information written in the file migration information, the secondstorage apparatus determines whether the file migration is allowablebased on a current load on the second storage apparatus itself and theincrement, the second storage apparatus notifies the determinationresult to the one of the plurality of first storage apparatuses that hasnotified the file migration information, and the one of the plurality offirst storage apparatuses determines whether to migrate the file to thesecond storage apparatus based on the determination result.

The other problems disclosed by the present invention and the solutionsthereof are made clear by the description of embodiments and the figuresof the present invention.

Advantageous Effects of Invention

According to the present invention, the reliability of the informationprocessing system can be improved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing the general configuration of an informationprocessing system 1.

FIG. 2 is a diagram showing the hardware configuration of a clientdevice 2.

FIG. 3 is a diagram showing the hardware configuration of a serverapparatus 3.

FIG. 4 is a diagram showing the hardware configuration of a storageapparatus 10.

FIG. 5 is a diagram showing the hardware configuration of a channelsubstrate 11.

FIG. 6 is a diagram showing the hardware configuration of a processorsubstrate 12.

FIG. 7 is a diagram showing the hardware configuration of a drivesubstrate 13.

FIG. 8 is a diagram showing basic functions of the storage apparatus 10.

FIG. 9 is a flowchart for explaining write processing S900.

FIG. 10 is a flowchart for explaining read processing S1000.

FIG. 11 is a diagram showing main functions of the client device 2.

FIG. 12 is a diagram showing main functions of a first server apparatus3 a and main information (data) managed in the first server apparatus 3a.

FIG. 13 is an example of a replication information management table 331.

FIG. 14 is an example of a file access log 332.

FIG. 15 is an example of a file migration policy table 333.

FIG. 16 is a diagram showing main functions of a second server apparatus3 b.

FIG. 17 is a diagram showing main functions of a second storageapparatus 10 b.

FIG. 18 is a diagram showing the general configuration of theinformation processing system 1.

FIG. 19 is an example of a file migration management table 110.

FIG. 20 is a diagram for explaining a file system configuration 2200.

FIG. 21 is a diagram for explaining the concept of inodes.

FIG. 22 is a diagram for explaining inode management table 2212.

FIG. 23 is a diagram for explaining inode management table 2212.

FIG. 24 is a diagram for explaining an inode management table 2212 inthe present embodiment.

FIG. 25 is a diagram for explaining the flow of the entire processing ofthe information processing system 1.

FIG. 26 is a diagram for explaining replication start processing S2700.

FIG. 27 is a flowchart for explaining replication start processingS2700.

FIG. 28 is a diagram for explaining stubbing candidate selectionprocessing S2900.

FIG. 29 is a flowchart for explaining stubbing candidate selectionprocessing S2900.

FIG. 30 is a diagram for explaining stubbing processing S3100.

FIG. 31 is a flowchart for explaining stubbing processing S3100.

FIG. 32 is a diagram for explaining metadata access processing S3900.

FIG. 33 is a flowchart for explaining metadata access processing S3900.

FIG. 34 is a diagram for explaining stubbed file entity referenceprocessing S4100.

FIG. 35 is a flowchart for explaining stubbed file entity referenceprocessing S4100.

FIG. 36 is a diagram for explaining stubbed file entity updateprocessing S4300.

FIG. 37 is a flowchart for explaining stubbed file entity updateprocessing S4300.

FIG. 38 is a flowchart for explaining migration determination processingS10030.

FIG. 39 is a flowchart for explaining re-evaluation processing formigration waiting to be processed.

FIG. 40 is a diagram for explaining the processing in case the migrationis determined not allowed to perform.

FIG. 41 is a flowchart for explaining the processing in case themigration is determined not allowed to perform.

FIG. 42 is a diagram for explaining the processing in case the migrationis determined not allowed to perform.

DESCRIPTION OF EMBODIMENTS

Hereinafter, description of the embodiments of the present invention isgiven with reference to the figures.

A general configuration of an information processing system 1 related tothe present embodiment is shown in FIG. 1. The information processingsystem 1 is configured of a plurality of systems (hereinafter alsoreferred to as Edges 50) which are respectively structured in aplurality of bases such as branch offices of a company, businessestablishments and the like and a system which is structured at a base(hereinafter also referred to as Core 51) such as a data center forperforming operation management of the respective Edges 50, provision ofapplication services and provision of the cloud service to therespective Edges 50 and other operations are communicatively coupled viaa communication network 7 such as WAN (Wide Area Network) and LAN (LocalArea Network).

The respective Edges 50 may be systems installed at the respective basesof the same company and may also be systems installed at the respectivebases of different companies.

A first server apparatus 3 a, a first storage apparatus 10 a, and aclient device 2 are installed in each of the Edges 50. Furthermore, afirst storage system 8 a is configured to include the first serverapparatus 3 a and the first storage apparatus 10 a. Obviously, thestorage system 8 a may be configured to include two apparatuses whichare the storage apparatus 10 a and the server apparatus 3 a, and mayalso be configured as a single storage apparatus having the functions ofboth the storage apparatus 10 a and the server apparatus 3 a.

Furthermore, a second server apparatus 3 b and a second storageapparatus 10 b are installed in the Core 51. Subsequently, a secondstorage system 8 b is configured including the second server apparatus 3b and the second storage apparatus 10 b. The storage system 8 b may beconfigured to include two apparatuses which are the storage apparatus 10b and the server apparatus 3 b, and may also be configured as a singlestorage apparatus including the functions of both the storage apparatus10 b and the server apparatus 3 b.

It should be noted that the location of each of the components in theinformation processing system 1 is not necessarily limited to what isshown in FIG. 1. Furthermore, in the explanation below, for simplifyingand rationalizing the explanation, the first server apparatus 3 a andthe second server apparatus 3 b are also collectively referred to asserver apparatuses 3. Furthermore, for example, the first storageapparatus 10 a and the second storage apparatus 10 b are alsocollectively referred to as storage apparatuses 10. The other componentsare also collectively referred to in a similar manner as needed.

In each of the Edges 50, a client device 2, the first server apparatus 3a, and the first storage apparatus 10 a perform information processingevery day. Subsequently, along with the performance of informationprocessing, new data such as files are generated and accumulated in thefirst storage apparatus 10 a every day.

While the details are explained later, for example, in a specified timeperiod such as at night after business hours of each of the Edges 50,these data are migrated to the second storage system 8 b in the Core 51in accordance with a policy (a file migration policy) stored in thefirst server apparatus 3 a of each of the Edges 50.

At the time of migration, the first storage apparatus 10 a transmits(replicates) a replication of the migration target data from the firststorage apparatus 10 a to the second storage apparatus 10 b, stores themetadata such as the storage location of those data in the secondstorage apparatus 10 b as a stub file in the first storage apparatus 10a, and also deletes the entity of the data migrated to the secondstorage apparatus 10 b from the first storage apparatus 10 a (stubbing).

The information processing system 1 by performing the above-mentionedmigration, allows various types of management operations, such as databackup that were respectively performed in each of the Edges 50, can beconsolidated in the Core 51, and thus various costs such as theabove-mentioned backup operation cost, the software cost, and theequipment cost can be reduced at each of the Edges 50.

The second storage system 8 b in the Core 51 stores the datarespectively transmitted from the respective Edges 50 in the secondstorage apparatus 10 b within the respective specified time period.

In the information processing system 1 related to the presentembodiment, before migrating these data to the second storage system 8b, the first storage system 8 a of each of the Edges 50 transmits filemigration information that has written therein information related tofile migration such as information for specifying the total amount ofdata to be transmitted and the deadline for migration to the secondstorage system 8 b.

Subsequently, the second storage system 8 b adjusts the timing for datamigration from each of the Edges 50 based on such as file migrationinformation transmitted from each of the Edges 50 and the load on theCore 51 at the current point of time. The second storage system 8 brepeatedly performs this data migration timing adjustment at a specifiedtiming.

As a result of the above-mentioned timing adjustment, the first storagesystem 8 a of the Edge 50 whose data migration is determined allowablestarts to transmit the data to the second storage system 8 b.Subsequently, the second storage system 8 b receives the datatransmitted from this first storage system 8 a and stores the data inthe second storage apparatus 10 b.

Meanwhile, the first storage system 8 a in the Edge 50 whose datamigration is determined to be at hold does not start data migration andwaits for the result of the next adjustment to be transmitted from thesecond storage system 8 b.

Furthermore, the first storage system 8 a for which data migration isnot determined in spite of waiting for data migration and which cannotperform data migration within the time period specified by the filemigration information suspends the data migration to the second storagesystem 8 b.

In this case, this first storage system 8 a, for example, may alsotransmit the file migration information to the second storage system 8 bagain after a specified time period (for example, twenty-four hourslater) and performs data migration to the second storage system 8 bafter data migration is determined allowable. This first storage system8 a may perform data migration to another storage system different fromthe second storage system 8 b (for example, a third storage system 8 cexplained later).

While the details are explained later, the information processing system1 performs the above-mentioned control to allow migration processing tobe performed while taking into consideration the information processingcapability of and the load on the Core 51. Subsequently, for example, byscheduling the data migration from each of the Edges 50 so as not toexceed the information processing capability of the Core 51, theoccurrence of failures and troubles in the Core 51 can be prevented andthe reliability of the information processing system 1 can be improved.In this way, the maintainability of the user data of each of the Edges50 can be improved.

Hereinafter, each of the components in the information processing system1 is sequentially explained.

The first server apparatus 3 a is, for example, a file storage apparatusincluding a file system which provides the function of data managementin units of files to the client device 2.

The first server apparatus 3 a may also be realized as a virtual machineby a virtualization control mechanism (a host OS type, a hypervisortype, and the like).

The second server apparatus 3 b is, for example, an apparatus (anarchive apparatus) functioning as a data library (an archive) for thefirst storage apparatus 10 a in the Edge 50. The second server apparatus3 b is implemented, for example, by using resources provided by thecloud service. The second server apparatus 3 b may also be implementedas a virtual machine by a virtualization control mechanism (a host OStype, a hypervisor type, and the like).

The client device 2 and the first server apparatus 3 a arecommunicatively coupled via a first communication network 5.Furthermore, the first server apparatus 3 a and the first storageapparatus 10 a are communicatively coupled via a first storage network 6a.

The first server apparatus 3 a and the second server apparatus 3 b arecommunicatively coupled via a second communication network 7.Furthermore, the second server apparatus 3 b and the second storageapparatus 10 b are communicatively coupled via a second storage network6 b.

The first communication network 5 and the second communication network 7are, for example, LAN (Local Area Network), WAN (Wide Area Network), theInternet, a public communication network, an exclusive line, and thelike.

The first storage network 6 a and the second storage network 6 b are,for example, LAN, WAN, SAN (Storage Area Network), the Internet, apublic telecommunication network, an exclusive line, and the like.

The communication performed via the first communication network 5, thesecond communication network 7, the first storage network 6 a, and thesecond storage network 6 b is performed in accordance with protocols,for example, TCP/IP, iSCSI (Internet Small Computer System Interface),Fibre Channel Protocol, FICON (Fibre Connection) (registered trademark),ESCON (Enterprise System Connection) (registered trademark), ACONARC(Advanced Connection Architecture) (registered trademark), FIBARC (FibreConnection Architecture) (registered trademark), and the like.

The client device 2 is an information processing apparatus (a computer)which utilizes a storage area provided by the first storage apparatus 10a via the first server apparatus 3 a. The client device 2 is, forexample, a personal computer, an office computer, a notebook computer, atablet mobile terminal, and the like. Furthermore, an Operating System,applications, and the like implemented by a software module (a filesystem, a kernel, a driver, and the like) function in the client device2.

The client device 2 transmits read requests and write requests to thefirst server apparatus 3 a for reading and writing the files requiredfor implementing the applications from and to the first storageapparatus 10 a.

The hardware configuration of the client device 2 is shown in FIG. 2.The client device 2 includes a processor 21, a volatile or non-volatilememory 22 (a RAM (Random Access Memory), a ROM (Read Only Memory), anNVRAM (Non Volatile RAM), and the like), a storage device 23 (an HDD(Hard Disk Drive), a semiconductor memory device (an SSD (Solid StateDrive) and others)), an input device 24 (a keyboard, a mouse, a touchscreen, and the like), an output device 25 (an LCD monitor, a printer,and the like), and a communication interface (also referred to as acommunication I/F 26) such as an NIC (an Network Interface Card)(hereinafter also referred to as a LAN adapter 261).

The first server apparatus 3 a is an information processing apparatuswhich provides information processing service to the client device 2 byutilizing the storage area provided by the first storage apparatus 10 aas a data storage location. The first server apparatus 3 a is configuredof computers such as a personal computer, a mainframe, an officecomputer, and the like, for example.

When accessing the storage area provided by the first storage apparatus10 a, in accordance with read requests and write requests transmittedfrom the client device 2, the first server apparatus 3 a transmits dataframes (hereinafter abbreviated to frames) including I/O requests (datawrite requests, data read requests, and the like) to the first storageapparatus 10 a via the first storage network 6 a. It should be notedthat the above-mentioned frames are Fibre Channel frames (FC frames (FC:Fibre Channel)), for example.

The second server apparatus 3 b is an information processing apparatuswhich provides the information processing service by utilizing thestorage area provided by the second storage apparatus 10 b. The secondserver apparatus 3 b is configured by using a personal computer, amainframe, an office computer, and the like. When accessing the storagearea provided by the second storage apparatus 10 b, the second serverapparatus 3 b transmits frames including I/O requests to the secondstorage apparatus 10 b via the second storage network 6 b.

Next, the hardware configuration of the server apparatus 3 is shown inFIG. 3.

The server apparatus 3 includes a processor 31, a volatile ornon-volatile memory 32 (a RAM, a ROM, an NVRAM, and the like), a storagedevice 33 (an HDD, a semiconductor storage device, and the like), aninput device 34 (a keyboard, a mouse, and the like), an output device 35(an LCD monitor, a printer, and the like), a communication interface(also referred to as a communication I/F 36) (an NIC (hereinafter alsoreferred to as a LAN adapter 361) and an HBA (hereinafter also referredto as an FC adapter 362), and the like), and a timer 37 configured byusing a timer circuit, an RTC, and the like.

Next, the hardware configuration of the storage apparatus 10 (the firststorage apparatus 10 a, the second storage apparatus 10 b) is shown inFIG. 4. The storage apparatus 10 is a disk array apparatus, for example.The storage apparatus 10 receives I/O requests transmitted from theserver apparatus 3 (the first server apparatus 3 a and the second serverapparatus 3 b), accesses the storage media in accordance with thereceived I/O requests, and transmits data or responses to the serverapparatus 3.

As shown in FIG. 4, the storage apparatus 10 includes one or morechannel substrates 11, one or more processor substrates 12 (MicroProcessors), one or more drive substrates 13, a Cache Memory 14, aShared Memory 15, an internal switch 16, a storage device 17, and amaintenance apparatus 18 (SVP: SerVice Processor). The channel substrate11, the processor substrate 12, the drive substrate 13, the Cache Memory14, and the Shared Memory 15 are communicatively coupled to one anothervia the internal switch 16.

The channel substrate 11 receives frames transmitted from the serverapparatus 3 and transmits the frames including responses (for example,read data, read completion reports, and write completion reports) of theprocessing (data I/O) related to the I/O requests included in thereceived frames to the server apparatus 3.

In response to the above-mentioned I/O requests included in framesreceived by the channel substrate 11, the processor substrate 12performs processing related to data transfer (high-speed, large-capacitydata transfer by using DMA (Direct Memory Access) and the like)performed among the channel substrate 11, the drive substrate 13, andthe Cache Memory 14. The processor substrate 12 performs transfer(delivery) of data between the channel substrate 11 and the drivesubstrate 13 (data which is read from the storage device 17 and data tobe written to the storage device 17) performed via the Cache Memory 14,staging of data which is stored in the Cache Memory 14 (reading datafrom the storage device 17), and destaging (writing data to the storagedevice 17) and the like.

The Cache Memory 14 is configured by using a RAM (Random Access Memory)capable of high-speed access. The data to be written to the storagedevice 17 (hereinafter also referred to as write data), the data readfrom the storage device 17 (hereinafter also referred to as read data),and the like are stored in the Cache Memory 14. Various types ofinformation used for controlling the storage apparatus 10 are stored inthe Shared Memory 15.

The drive substrate 13 performs communication with the storage device 17for reading data from the storage device 17 and writing data to thestorage device 17. The internal switch 16 is configured by using ahigh-speed crossbar Switch, for example. It should be noted that thecommunication performed via the internal switch 16 is performed inaccordance with protocols, for example, Fibre Channel, iSCSI, TCP/IP,and the like.

The storage device 17 is configured of a plurality of storage drives171. The storage drives 171 are hard disk drives of, for example, SAS(Serial Attached SCSI), SATA (Serial ATA), FC (Fibre Channel), PATA(Parallel ATA), SCSI, and other types, semiconductor storage devices(SSDs), and the like.

The storage device 17 provides the storage area of the storage device 17to the server apparatus 3 in units of logical storage areas provided bycontrolling the storage drives 171, for example, by RAID (RedundantArrays of Inexpensive (or Independent) Disks) and other methods. Thislogical storage area is, for example, a storage area of a logical device(an LDEV 172 (LDEV: Logical Device)) configured by using a RAID group(Parity Group).

Furthermore, the storage apparatus 10 provides the logical storage areaconfigured by using the LDEV 172 (hereinafter also referred to as an LU(Logical Unit, Logical Volume) 173) to the server apparatus 3. Anidentifier (hereinafter also referred to as an LUN) is attached to eachof the LUs 173. The storage apparatus 10 manages the correspondence(relationship) between the LUs 173 and the LDEVs 172. Based on theabove-mentioned correspondence, the storage apparatus 10 identifies theLDEVs 172 corresponding to the LUs 173 and identifies the LUs 173corresponding to the LDEVs 172.

The hardware configuration of the channel substrate 11 is shown in FIG.5. The channel substrate 11 includes an external communication interface(hereinafter referred to as an external communication I/F 111) having aport (a communication port) for communicating with the server apparatus3, a processor 112 (including a frame processing chip and a frametransfer chip), a memory 113, an internal communication interface(hereinafter referred to as an internal communication I/F 114) having aport (a communication port) for communicating with the processorsubstrate 12, and the like.

The external communication I/F 111 is configured by using an NIC(Network Interface Card), an HBA (Host Bus Adaptor), and the like. Theprocessor 112 is configured by using a CPU (Central Processing Unit), anMPU (Micro Processing Unit), and the like. The memory 113 is a RAM(Random Access Memory) or a ROM (Read Only Memory). A microprogram isstored in the memory 113. Various functions provided by the channelsubstrate 11 are implemented by the processor 112 reading theabove-mentioned microprogram from the memory 113 and executing the same.The internal communication I/F 114 communicates with the processorsubstrate 12, the drive substrate 13, the Cache Memory 14, and theShared Memory 15 via the internal switch 16.

A hardware configuration of the processor substrate 12 is shown in FIG.6. The processor substrate 12 comprises an internal communicationinterface (hereinafter referred to as an internal communication I/F121), a processor 122, and a memory 123 (local memory) having highaccess performance (capable of high-speed access) from the processor 122compared with the Shared Memory 15. A microprogram is stored in thememory 123. Various functions provided by the processor substrate 12 areimplemented by the processor 122 reading the above-mentionedmicroprogram from the memory 123 and executing the same.

The internal communication I/F 121 communicates with the channelsubstrate 11, the drive substrate 13, the Cache Memory 14, and theShared Memory 15 via the internal switch 16. The processor 122 isconfigured by using a CPU, an MPU, a DMA (Direct Memory Access), and thelike. The memory 123 is a RAM or a ROM. The processor 122 can accessboth the memory 123 and the Shared Memory 15.

The hardware configuration of the drive substrate 13 is shown in FIG. 7.The drive substrate 13 includes an internal communication interface(hereinafter referred to as an internal communication I/F 131), aprocessor 132, a memory 133, and a drive interface (hereinafter referredto as a drive I/F 134). A microprogram is stored in the memory 133.Various functions provided by the drive substrate 13 are implemented bythe processor 132 reading the above-mentioned microprogram from thememory 133 and executing the same. The internal communication I/F 131communicates with the channel substrate 11, the processor substrate 12,the Cache Memory 14, and the Shared Memory 15 via the internal switch16. The processor 132 is configured by using a CPU, an MPU, and thelike. The memory 133 is, for example, a RAM or a ROM. The drive I/F 134performs communication with the storage device 17.

The maintenance apparatus 18 shown in FIG. 4 performs the control of therespective components and the status monitoring of the storage apparatus10. The maintenance apparatus 18 is a personal computer, an officecomputer, and the like. The maintenance apparatus 18 performscommunication as needed with the components in the storage apparatus 10such as the channel substrate 11, the processor substrate 12, the drivesubstrate 13, the Cache Memory 14, the Shared Memory 15, the internalswitch 16, via communication means such as the internal switch 16 andthe exclusively prepared LAN, acquires operation information from therespective components, and provides the information to the managementapparatus 19. Furthermore, the maintenance apparatus 18 performssetting, control, and maintenance (including the installation and updateof software) of the respective components based on the controlinformation and operation information transmitted from the managementapparatus 19.

The management apparatus 19 is a computer which is communicativelycoupled to the maintenance apparatus 18 in the storage apparatus 10 viaLAN and the like. The management apparatus 19 comprises a user interface(a GUI (Graphical User Interface), a CLI (Command Line Interface), andthe like) for controlling and monitoring the storage apparatus 10.

It should be noted that the management apparatus 19 may be coupled tothe server apparatus 3 besides the case of being coupled to the storageapparatus 10. In this case, the management apparatus 19 may also performthe control of the respective components and the status monitoring ofthe server apparatus 3 as well as performing the control of therespective components and the status monitoring of the storage apparatus10. For example, the management apparatus 19 may also perform managementand control related to the file system.

Next, the basic functions of the storage apparatus 10 are shown in FIG.8. The storage apparatus 10 includes an I/O processing unit 811. The I/Oprocessing unit 811 includes a data write processing unit 8111 whichperforms processing related to data write to the storage device 17 inthe storage apparatus 10 and a data read processing unit 8112 whichperforms processing related to data read from the storage device 17 inthe storage apparatus 10.

It should be noted that the I/O processing unit 811 is implemented bythe hardware which the channel substrate 11, the processor substrate 12,or the drive substrate 13 includes or by the processor 112, theprocessor 122, or the processor 132 reading and executing themicroprogram stored in the memory 113, the memory 123, or the memory133.

Next, basic processing (hereinafter also referred to as the writeprocessing S900) performed by the data write processing unit 8111 in theI/O processing unit 811 in case the storage apparatus 10 receives aframe including a data write request from the server apparatus 3 isexplained with reference to the flowchart shown in FIG. 9. It should benoted that the letter “S” affixed to a reference sign is an abbreviationfor step.

As shown in FIG. 9, firstly, a frame of a data write request transmittedfrom the server apparatus 3 is received by the channel substrate 11 inthe storage apparatus 10 (S911, S912). Upon reception of the frameincluding the data write request from the server apparatus 3, thechannel substrate 11 reports the reception to the processor substrate 12(S913).

When receiving the above-mentioned notification from the channelsubstrate 11 (S921), the processor substrate 12 generates a drive writerequest based on the data write request in the relevant frame, storesthe write data in the Cache Memory 14, and also responds to the channelsubstrate 11 by a reception notification of the above-mentionednotification (S922). Furthermore, the processor substrate 12 transmitsthe generated drive write request to the drive substrate 13 (S923).

Meanwhile, when receiving the above-mentioned response from theprocessor substrate 12, the channel substrate 11 transmits a completionreport to the server apparatus 3 (S914). The server apparatus 3 receivesthe completion report from the channel substrate 11 (S915).

When receiving the drive write request from the processor substrate 12,the drive substrate 13 registers the received drive write request in thewrite processing queue (S924).

The drive substrate 13 reads the drive write request from the writeprocessing queue as needed (S925), reads the write data specified by theread drive write request from the Cache Memory 14, and writes the readwrite data in the storage device (storage drives 171) (S926).Subsequently, the drive substrate 13 notifies a report (completionreport) that the data write related to the drive write request iscompleted to the processor substrate 12 (S927).

The processor substrate 12 receives the completion report transmittedfrom the drive substrate 13 (S928).

Next, the I/O processing (hereinafter also referred to as the readprocessing S1000) performed by the data read processing unit 8112 in theI/O processing unit 811 of the storage apparatus 10 in case the storageapparatus 10 receives a frame including a data read request from theserver apparatus 3 is explained with reference to the flowchart shown inFIG. 10.

As shown in FIG. 10, firstly, a frame transmitted from the serverapparatus 3 is received by the channel substrate 11 in the storageapparatus 10 (S1011, S1012).

When receiving the frame including the data read request from the serverapparatus 3, the channel substrate 11 notifies the reception to thedrive substrate 13 (S1013).

When receiving the above-mentioned notification from the channelsubstrate 11 (S1014), the drive substrate 13 reads the data specified bythe data read request (for example, specified by an LBA (Logical BlockAddress)) included in the relevant frame from the storage device(storage drives 171) (S1015). It should be noted that the readprocessing from the storage device 17 (S1015) is omitted when the readdata exists in the Cache Memory 14 (in case of a cache hit).

The processor substrate 12 writes the data which is read by the drivesubstrate 13 in the Cache Memory 14 (S1016). Subsequently, the processorsubstrate 12 transfers the data written in the Cache Memory 14 to thechannel substrate 11 as needed (S1017).

When receiving the read data transmitted as needed from the processorsubstrate 12, the channel substrate 11 transmits the data to the serverapparatus 3 sequentially (S1018). When the transmission of the read datais completed, the channel substrate 11 transmits a completion report tothe server apparatus 3 (S1019). The server apparatus 3 receives the readdata and the completion report (S1020, S1021).

Next, FIG. 11 shows the main functions of the client device 2. Theclient device 2 includes functions of an application 211, a file system212, a kernel/driver 213, and a protocol client 214. It should be notedthat these functions are implemented by the processor 21 in the clientdevice 2 reading and executing the programs stored in the memory 22 andthe storage device 23.

The application 211 is a program executed to provide an informationprocessing service to the users and the like of the Edges 50. The clientdevice 2 executes I/O operations such as transmitting data I/O requests(read requests and write requests) in units of files to the firststorage system 8 a by the client device 2 executing the application 211.

For the client device 2, the file system 212 implements the I/O functionin units of files or in units of directories to the logical volumes(LUs) 173. The file system 212 is, for example, FAT (File AllocationTable), NTFS, HFS (Hierarchical File System), ext2 (second extended filesystem), ext3 (third extended file system), ext4 (fourth extended filesystem), UDF (Universal Disk Format), HPFS (High Performance Filesystem), JFS (Journaled File System), UFS (Unix File System), VTOC(Volume Table Of Contents), XFS, NFS, CIFS, and the like.

The kernel/driver 213 is implemented by running a kernel module and adriver module configuring the software of the operating system. Forsoftware executed in the client device 2, programs for implementing thebasic functions of the operating system are included in the kernelmodule. The basic functions of the operating system are such as processmanagement, process scheduling, storage area management, and handling ofinterrupt requests from the hardware. Programs and the like for thekernel module to communicate with the hardware configuring the clientdevice 2 and the peripherals used by coupling to the client device 2 areincluded in the driver module.

The protocol client 214 performs processing related to protocols such asNFS and CIFS in cooperation with the protocol server 320 performed inthe first server apparatus 3 a when the client device 2 performs I/Ooperations such as read requests and write requests for the firststorage system 8 a.

Next, the main functions of the first server apparatus 3 a and the maininformation (data) managed in the first server apparatus 3 a are shownin FIG. 12. The first server apparatus 3 a includes functions of a filesystem 312, a data replication/migration processing unit 314, a fileaccess log acquisition unit 317, a kernel/driver 318, a coordinatorclient 319, and a protocol server 320.

It should be noted that these functions are implemented by the hardwareof the first server apparatus 3 a or the processor 31 in the firstserver apparatus 3 a reading and executing the programs stored in thememory 32. It should be noted that the functions of the data operationrequest acceptance unit 313, the data replication/migration processingunit 314, and the file access log acquisition unit 317 may beimplemented as functions of the file system 312 and may also be achievedas functions independent of the file system 312.

Furthermore, the first server apparatus 3 a manages (stores) information(data) such as a replication information management table 331, a fileaccess log 332, and a file migration policy table 333. These informationare stored, for example, in the memory 32 and the storage device 33 inthe first server apparatus 3 a.

For the client device 2, the file system 312 provides an I/O functionfor files and directories managed by the logical volumes (LUs) 173provided by the first storage apparatus 10 a. The file system 312 is,for example, FAT (File Allocation Table), NTFS, HFS (Hierarchical FileSystem), ext2 (second extended file system), ext3 (third extended filesystem), ext4 (fourth extended file system), UDF (Universal DiskFormat), HPFS (High Performance File system), JFS (Journaled FileSystem), UFS (Unix File System), VTOC (Volume Table Of Contents), XFS,NFS, CIFS, and the like.

The data replication/migration processing unit 314 performs varioustypes of file-related control with the other server apparatus 3 (thesecond server apparatus 3 b) or with the storage apparatuses 10 (thefirst storage apparatus 10 a, the second storage apparatus 10 b). Forexample, the data replication/migration processing unit 314 performstransmission/reception of data (including metadata and entity of files),transmission/reception of control information (including flags andtables), management of the various tables, and the like.

Furthermore, the data replication/migration processing unit 314 performsreplication control, update control for replication files (files whosereplication stored in the first storage apparatus 10 a is also stored inthe second storage apparatus 10 b), reference control for replicationfiles, synchronization control, access control for metadata, referencecontrol for the entity of files, recall control, update control for theentity of stubbed files, and other types of processing, which areexplained later.

Replication means that the replication of the metadata and entity of thedata of files and directories to be stored in the first storageapparatus 10 a is also stored in another storage apparatus 10 (forexample, the second storage apparatus 10 b).

Stubbing means that the metadata of files and directories is managed(stored) in the first storage apparatus 10 a while the entity of thedata of files and directories is not managed (stored) in the firststorage apparatus 10 a and is managed (stored) only in another storageapparatus 10 (for example, the second storage apparatus 10 b).

A stub indicates the metadata which remains in the first storageapparatus 10 a in the above-mentioned case. When the first serverapparatus 3 a receives a read request or a write request to a stubbedfile or directory for which an entity of the file or directory becomesnecessary, the entity of the file or directory is transmitted from theother storage apparatus 10 to the first storage apparatus 10 a(hereinafter also referred to as recall).

Furthermore, after the metadata and entity of a file to be stored in thefirst storage apparatus 10 a is replicated to the second storageapparatus 10 b, in this case the file is stubbed at a specified timing,such processing is also referred to as migration. The specified timingcan be, for example, the case where the first server apparatus 3 adetects that the remaining storage capacity of the first storageapparatus 10 a falls below a specified value, the case where the firstserver apparatus 3 a detects that the usage rate of the storage capacityof the first storage apparatus 10 a exceeds a specified rate, the casewhere the first server apparatus 3 a detects a file which is notaccessed within a specified period of time after replication, and othercases. Or the files may be stubbed following the replication of thefiles stored in the first storage apparatus 10 a to the second storageapparatus 10 b.

Furthermore, at a specified time every day for example (a specified timelate at night for example), in accordance with a file migration policywritten in the file migration policy table 333 explained later, the datareplication/migration processing unit 314 extracts a file to be migratedto the second storage system 8 b from the files stored in the firststorage apparatus 10 a.

Subsequently, the data replication/migration processing unit 314transmits file migration information including information such as thetotal data amount of the extracted migration target file and the timeperiod for transmission (Window) to the coordinator client 319, andtransmits the extracted migration target file to the second storagesystem 8 b under control of the coordinator client 319.

Specifically speaking, when receiving a notification from thecoordinator client 319 that migration can be performed, the datareplication/migration processing unit 314 starts to transmit themigration target file to the second storage system 8 b.

Furthermore, when receiving a notification from the coordinator client319 that migration is to be at standby, the data replication/migrationprocessing unit 314 does not transmit the migration target file to thesecond storage system 8 b and remains at standby.

Furthermore, when receiving a notification from the coordinator client319 that migration cannot be performed, the data replication/migrationprocessing unit 314 suspends the transmission of the file to the secondstorage system 8 b.

When acquiring the file migration information including information suchas the total data amount of the migration target file and theinformation for specifying the time period for transmission (Window)from the data replication/migration processing unit 314, the coordinatorclient 319 writes the file migration information to the file migrationmanagement table 110 which is stored in the shared LU 173 of the secondstorage apparatus 10 b.

The shared LU 173 is a storage area configured to allow data read andwrite from the first server apparatuses 3 a of all the Edges 50.

When the first server apparatus 3 a of each of the Edges 50 writes thefile migration information describing the total data amount of each ofthe migration target files, the time period for transmission, and otherinformation in the shared LU 173, the coordinator server 346 executed bythe second server apparatus 3 b determines whether the migrationrequested by each of the Edges 50 can be performed or not based on theabove-mentioned file migration information and the load on the secondstorage apparatus 10 b. Subsequently, the coordinator server 346 writesthe determination result for each of the requests to the file migrationmanagement table 110.

The coordinator client 319 of each of the Edges 50 refers to the filemigration management table 110 at a specified timing respectively andacquires the determination result on whether migration can be performedor not. Subsequently, the coordinator client 319 notifies thedetermination result to the data replication/migration processing unit314.

The kernel/driver 318 is implemented by executing the kernel module andthe driver module configuring the software of the operating system.Programs for implementing the basic functions of the operating systemfor the software executed in the first server apparatus 3 a such asprocess management, process scheduling, storage area management, andhandling of interrupt requests from the hardware are included in thekernel module. Programs for the kernel module to communicate with thehardware configuring the first server apparatus 3 a and the peripheralsused by connection to the first server apparatus 3 a are included in thedriver module.

When a file stored in the logical volume (LU) 173 in the storageapparatus 10 is accessed (updating a file (Write, Update), reading afile (Read), opening a file (Open), closing a file (Close), and thelike), the file access log acquisition unit 317 stores the information(hereinafter also referred to as an access log) indicating the contents(history) of the access as an file access log 332 while providing atimestamp based on the date and time information acquired from the timer37.

The protocol server 320 executes processing related to the protocolssuch as NFS and CIFS in cooperation with the protocol client 214performed in the client device 2 when the client device 2 performs I/Ooperations such as read requests and write requests for the firststorage system 8 a.

Next, an example of the replication information management table 331 isshown in FIG. 13. A host name 3311 to be the replication destination(for example, a network address such as an IP address of the secondserver apparatus 3 b) and a threshold 3312 (hereinafter also referred toas a stubbing threshold) used for determining whether to performstubbing or not are set in the replication information management table331.

Although a threshold for performing stubbing in a case the remainingstorage capacity of the first storage apparatus 10 a falls below 200 GBis set as the stubbing threshold 3312 in the example shown in FIG. 13,the stubbing threshold 3312 can be set in various ways.

For example, a time period (for example, ten days) may be set as thestubbing threshold and a file which has not been read or updated withinthe period is stubbed. Or an extension of a specific file may be set asthe stubbing threshold and stubbing is performed for the file with theextension.

Next, an example of the file access log 332 is shown in FIG. 14. Anaccess log configured of one or more records each configured ofrespective items of an access date and time 3351, a file name 3352, anda user ID 3353 is recorded in the file access log 332.

Among these items, the date and time on which the file or directory isaccessed is set in the access date and time 3351. The file name ordirectory name of the access target file or directory is set in thefilename 3352. The user ID of the user that has accessed the file ordirectory is set in the user ID 3353.

Next, an example of the file migration policy table 333 is shown in FIG.15. The file migration policy table 333 includes a “TARGET FILE” fieldand a “WINDOW” field.

The information indicating the migration target file is written in the“TARGET FILE” field. The information indicating the period of time inwhich the migration target file is migrated is written in the “WINDOW”field. Although migration may be performed at any point of time withinthe time period shown in the “WINDOW” field, the ending time of the timeperiod in the “WINDOW” field is the time limit for migration.

The example shown in FIG. 15 shows that, among the files stored in thefirst storage apparatus 10 a, the entity of a synchronization necessityfile and the metadata of the synchronization necessity file are bothmigrated to the second storage apparatus 10 b between 10 pm and 6 am.

While the details are explained later, an entity of the synchronizationnecessity file indicates the entity of a file whose entitysynchronization necessity flag is set (“1”) in the inode managementtable 2212. Furthermore, a metadata of the synchronization necessityfile indicates the metadata of a file whose metadata synchronizationnecessity flag is set (“1”) in the inode management table 2212.

Furthermore, the “WINDOW” field in the file migration policy table 333is the information which specifies the time period in which migration isperformed (the point of time at which migration may be started and thetime limit for migration). This time period is appropriately set by thesystem administrator in accordance with the business configuration andthe like of the Edge 50. For example, the example shown in FIG. 15indicates that, as the period from 10 pm until 6 am is outside businesshours for the Edge 50 every day, the entity and metadata of the fileupdated or generated in the Edge 50 on the latest business day ismigrated to the Core 51 during this period.

The contents written in the file migration policy table 333 may also bein various forms in addition to what is shown in FIG. 15. For example, aspecific extension attached to a file, stored in the first storageapparatus 10 a is written in advance, and this file with this extensionmay be made the target of migration.

Next, the main functions of the second server apparatus 3 b are shown inFIG. 16. The second server apparatus 3 b includes functions of a filesystem 342, a data replication/migration processing unit 344, akernel/driver 345, a coordinator server 346, and a load acquisition unit347. It should be noted that the function of the datareplication/migration processing unit 344 may be implemented as afunction of the file system 342 or may be implemented as an independentfunction from the file system 342.

The file system 342 uses the logical volumes (LU) 173 provided by thesecond storage apparatus 10 b to provide an I/O function for the logicalvolumes (LUs or virtual LUs) in units of files or directories to thefirst server apparatus 3 a. The file system 342 is, for example, FAT,NTFS, HFS, ext2, ext3, ext4, UDF, HPFS, JFS, UFS, VTOC, XFS, NFS, CIFS,and the like.

The data replication/migration processing unit 344 performs theprocessing related to data migration and replication with the firstserver apparatus 3 and the second storage apparatus 10 b. For example,when a migration target file is transmitted from the first serverapparatus 3 a, the data replication/migration processing unit 344receives the file and writes the file to a specified storage area of thesecond storage apparatus 10 b.

The kernel/driver 345 is implemented by executing the kernel module andthe driver module configuring the software of the operating system. Forthe software performed in the second server apparatus 3 b, programs forimplementing basic functions of the operating system such as processmanagement, process scheduling, storage area management, and handling ofinterrupt requests from hardware are included in the kernel module.Programs for the kernel module to communicate with the hardwareconfiguring the second server apparatus 3 b and the peripherals used bycoupling to the second server apparatus 3 b are included in the drivermodule.

The coordinator server 346 refers to the file migration management table110 stored in the shared LU 173 in the second storage apparatus 10 b, ata specified timing, and determines whether migration can be performed ornot so that the load on the second storage apparatus 10 b does notexceed the processing ability based on each file migration informationstored in the file migration management table 110 and the current loadon the second storage apparatus 10 b. Subsequently, the coordinatorserver 346 associates the determination result to each file migrationinformation and writes the result in the file migration management table110.

The load acquisition unit 347 acquires the current load on the secondstorage apparatus 10 b. This load on the second storage apparatus 10 bis the load when the second storage apparatus 10 b receives a migrationtarget file from the first storage system 8 a of each of the Edges 50and performs processing for storing the file in the storage device 17and the load when the second storage apparatus 10 b reads the filestored in the storage device 17 and transmits the file to the firststorage system 8 a.

The load acquisition unit 347, for example, acquires a reception ratewhen the second storage apparatus 10 b receives the file transmittedfrom the first storage system 8 a (the transfer data amount per unittime) as a load from the kernel/driver 345. For example, when the secondstorage apparatus 10 b, while receiving a file from the first storagesystem 8 a which transmits data of no megabytes per second, receives afile from the other first storage system 8 a with a communication rateof 200 megabytes per second, the load on the second storage apparatus 10b is assumed to be 300 megabytes per second.

Next, a configuration of the second storage apparatus 10 b is shown inFIG. 17. The second storage apparatus 10 b includes an I/O processingunit 811 b. The function of the I/O processing unit 811 b is the same asthe function of the I/O processing unit 811 a which the first storageapparatus 10 a has.

Furthermore, the second storage apparatus 10 b includes a plurality ofLUs 173. A file transmitted from the first storage system 8 a of each ofthe Edges 50 to the second storage system 8 b is written only to the LU173 which is respectively assigned to each of the first storage systems8 a. Subsequently, the first storage system 8 a of each of the Edges 50can read the file only from the LU 173 assigned thereto.

Meanwhile, the second storage apparatus 10 b includes a shared LU 173for which read and write is enabled from the first storage systems 8 aof all the Edges 50. The file migration management table 110 is storedin this shared LU 173.

FIG. 18 shows a state in which the files transmitted from the firststorage systems 8 a of the respective Edges 50 to the second storagesystem 8 b are written only to the LUs 173 respectively assigned to therespective Edges 50 and, at the same time, read and write can beperformed for the file migration management table 110 stored in theshared LU 173 from the first storage systems 8 a of all the Edges 50.

It should be noted that instead of storing the file migration managementtable 110 in the shared LU 173, a part of the storage area of the LU 173assigned to a specific Edge 50 is set accessible also from the otherEdges 50 and that the file migration management table 110 is stored inthis storage area.

Next, an example of the file migration management table 110 is shown inFIG. 19. The file migration management table 110 includes a “NUMBER”field, a “COR (Cloud On Ramp)” field, a “JOB” field, a “WINDOW” field, a“TOTAL TRANSFER AMOUNT” field, an “EFFECTIVE PERFORMANCE” field, a“READ/WRITE” field, a “REMAINING CACHE” field, and a “RESULT” field. Itshould be noted that the “NUMBER” field is described as “#” in FIG. 19.

Among the values written in these respective fields, the values writtenin the “COR” field, the “JOB” field, the “WINDOW” field, the “TOTALTRANSFER AMOUNT” field, the “EFFECTIVE PERFORMANCE” field, the“READ/WRITE” field, and the “REMAINING CACHE” field are those written inthe file migration information transmitted from the first serverapparatus 3 a. Furthermore, the value written in the “NUMBER” field isthe value generated by the second storage apparatus 10 b or the secondserver apparatus 3 b. Furthermore, the value written in the “RESULT”field is the value written by the second server apparatus 3 b.

An identification number for uniquely identifying the file migrationinformation written to the file migration management table 110 by thefirst server apparatus 3 a of each of the Edges 50 is written in the“NUMBER” field. The number written in this “NUMBER” field is provided,for example, by the second storage apparatus 10 b when the secondstorage apparatus 10 b writes in the file migration management table110, the file migration information transmitted from the first serverapparatus 3 a.

Identification information of the first server apparatus 3 a which hastransmitted the file migration information is written in the “COR”field.

Information, provided by the first server apparatus 3 a which hastransmitted the file migration information, for identifying the filemigration information is written in the “JOB” field.

Information for specifying the time period for performing migrationwhich is the target of the file migration information (the point of timeat which migration may be started and the deadline for migration) iswritten in the “WINDOW” field. For example, in case of the filemigration information whose “#” field in FIG. 19 is 1, it is writtenthat the migration being the target of the file migration information isperformed between 10 pm and 6 am next morning.

The total data amount of files transmitted to the second storageapparatus 10 b by migration being the target of the file migrationinformation is written in the “TOTAL TRANSFER AMOUNT” field.

The data transfer rate in a case a file is transmitted to the secondstorage apparatus 10 b by migration being the target of the filemigration information is written in the “EFFECTIVE PERFORMANCE” field.This data transfer rate may be an actual measured value in the past forexample, and may also be an upper limit value or a contract value of thedata transfer rate of the second communication network 7 between thefirst server apparatus 3 a and the second server apparatus 3 b.

In the “READ/WRITE” field, W is written in a case of writing the filetransmitted by the first server apparatus 3 a to the second storageapparatus 10 b, and R is written in a case the first server apparatus 3a reads the file stored in the second storage apparatus 10 b.Specifically speaking, W is written in the case of migration from thefirst storage system 8 a to the second storage system 8 b. Meanwhile, Ris written in the case of recall which occurs in a case such as when thestubbed file stored in the first storage system 8 a is updated and othercases.

The remaining storage capacity of the first storage apparatus 10 a in acase the first server apparatus 3 a transmits file migration informationis written in the “REMAINING CACHE” field. The usage rate of the storagecapacity of the first storage apparatus 10 a may also be written in this“REMAINING CACHE” field.

The result of the determination performed by the coordinator server 346as to whether migration can be performed or not based on each filemigration information stored in the file migration management table 110,and the processing ability and the load of the second storage apparatus10 b is written in the “RESULT” field. The result of the determinationis “ALLOWED”, “STANDBY”, or “NOT ALLOWED”.

The case of “ALLOWED” indicates that the migration being the target ofthe file migration information can be performed at the current point oftime. In this case, the first server apparatus 3 a which has writtenthis file migration information to the file migration management table110 starts to execute the migration.

The case of “STANDBY” indicates that the migration being the target ofthe file migration information cannot be performed at the current pointof time. In this case, the first server apparatus 3 a which has writtenthis file migration information to the file migration management table110 is at a standby for executing the migration.

The case of “NOT ALLOWED” indicates that the migration being the targetof the file migration information cannot be completed within the periodof time written in the “WINDOW” field. In this case, the first serverapparatus 3 a which has written this file migration information to thefile migration management table 110 suspends the execution of themigration to the second storage apparatus 10 b.

[File System]

Next, the configuration of the file system 312 in the first serverapparatus 3 a is explained in detail. It should be noted that the filesystem 342 in the second server apparatus 3 b has the same configurationas the file system 312 in the first server apparatus 3 a.

FIG. 20 is an example of the data structure (hereinafter also referredto as a file system structure 2200) which the file system 312 manages inthe logical volume 173. The file system configuration 2200 includes therespective storage areas of super block 2211, inode management table2212, and data block 2213 in which the entity of the file (data) isstored.

Information relating to the file system 312 (the capacity of the storagearea which the file system handles, the used amount, the free capacity,and the like) is stored in the super block 2211. The super block 2211 isbasically set for each disk partition (partition set in the logicalvolume 173). Concrete examples of the above-mentioned information storedin the super block 2211 are the number of data blocks in the partition,the size of a data block, the number of free blocks, the number of freeinodes, the number of mounts in the relevant partition, the elapsed timesince the latest consistency check, and the like.

Management information of a file or directory stored in the logicalvolume 173 (hereinafter also referred to as an inode) is stored in theinode management table 2212. The file system 312 makes one file ordirectory correspond to one inode and manages the above. Among theinodes, those including only information relating to a directory isreferred to as a directory entry. When accessing a file, the data blockof the access target file is accessed with reference to the directoryentry. For example, when accessing a file “/home/user-01/a.txt”, forexample, the data block of the access target file is accessed bytracking the inode numbers and directory entries in the order indicatedby arrows (“2”→“10”→“15”→“100”) as shown in FIG. 21.

Next, the concept of an inode in a common file system (for example, afile system of an operating system of the UNIX (registered trademark)series) is shown in FIG. 22. Furthermore, an example of the inodemanagement table 2212 is shown in FIG. 23.

As shown in FIG. 22 and FIG. 23, an inode includes information such asinode number 2511 which is an identifier for distinguishing individualinodes, owner 2512 of the relevant file or directory, access right 2513which is set as to the relevant file or directory, file size 2514 of therelevant file or directory, last update date and time 2515 of therelevant file or directory, parent directory 2516 of the directory whichis set in the case the relevant inode is a directory entry, childdirectory 2517 of the directory which is set incase the relevant inodeis a directory entry, and information for identifying the data block inwhich the entity of the data of the relevant file is stored (hereinafteralso referred to as a block address 2518).

Furthermore, as shown in FIG. 24, in addition to the contents of theinode management table 2212 in a common file system shown in FIG. 23,the file system 312 further manages stubbing flag 2611, metadatasynchronization necessity flag 2612, entity synchronization necessityflag 2613, replication flag 2614, link destination 2615 accompanying theinode management table 2212.

It should be noted that, in the case the replication of the metadata ofthe file stored in the first storage apparatus 10 a (the metadataincluding the various types of additional information shown in FIG. 24)is also stored in the second storage apparatus 10 b (in the case a fileis replicated) by a management method by replication or the managementmethod by stubbing and when the metadata of one of the apparatuses isupdated by the synchronization processing S3700 explained later, theupdate is notified to the other apparatus. According to this method, theconsistency of the contents between the metadata of the first storageapparatus 10 a and the metadata of the second storage apparatus 10 b issecured.

In FIG. 24, information indicating whether the file or directorycorresponding to the relevant inode is stubbed or not is set in thestubbing flag 2611. When the file or directory corresponding to therelevant inode is stubbed, “1” is set in the stubbing flag 2611 while“0” is set in the stubbing flag 2611 when the file or directory is notstubbed.

Information indicating whether it is necessary or not to synchronize (tomatch the contents of) the metadata of the file or directory of thefirst storage apparatus 10 a being the replication source with themetadata of the file or directory of the second storage apparatus 10 bbeing the replication destination is set for the metadatasynchronization necessity flag 2612. If it is necessary to synchronizethe metadata, “1” is set in the metadata synchronization necessity flag2612 while “0” is set in the metadata synchronization necessity flag2612 when synchronization is not necessary.

When the file of the first storage apparatus 10 a is replicated to thesecond storage apparatus 10 b and when the contents of the metadata ofthis file is updated only in the first storage apparatus 10 a, themetadata stored in the first storage apparatus 10 a and the metadatastored in the second storage apparatus 10 b become inconsistent. In suchcase, “1” is set in the metadata synchronization necessity flag 2612.

Information indicating whether it is necessary or not to synchronize (tomatch the contents) the entity of the data of the file of the firststorage apparatus 10 a which is the replication source with the entityof the data of the file of the second storage apparatus 10 b which isthe replication destination is set in the entity synchronizationnecessity flag 2613. When it is necessary to synchronize the entity ofthe data of the file, “1” is set for the entity synchronizationnecessity flag 2613 while “0” is set in the entity synchronizationnecessity flag 2613 when synchronization is not necessary.

When the file of the first storage apparatus 10 a is replicated to thesecond storage apparatus 10 b and when the entity of this file isupdated only in the first storage apparatus 10 a, the entity of the filestored in the first storage apparatus 10 a and the entity of the filestored in the second storage apparatus 10 b become inconsistent. In suchcase, “1” is set in the entity synchronization necessity flag 2613.

The metadata synchronization necessity flag 2612 and the entitysynchronization necessity flag 2613 are referred to as needed in thesynchronization processing S3700 which is explained later. When eitherthe metadata synchronization necessity flag 2612 or the entitysynchronization necessity flag 2613 is set to “1”, the metadata orentity of the first storage apparatus 10 a and the metadata or entity ofthe second storage apparatus 10 b being a replication thereof aresynchronized.

Information indicating whether or not the file or directorycorresponding to the inode is currently a target of management by thereplication management method which is explained later is set for thereplication flag 2614. “1” is set in the replication flag 2614 when thefile corresponding to the relevant inode is currently a target ofmanagement by the replication management method while “0” is set in thereplication flag 2614 when the file is not a target of management byreplication.

In a case a file corresponding to the inode is managed by thereplication management method explained later, information indicatingthe replication destination of the file (for example, the path name foridentifying a storage destination, the identifier of a RAID group, ablock address, a URL (Uniform Resource Locator), an LUN and others) isset in the link destination 2615.

=Description of Processing=

Next, the processing performed in the information processing system 1 isexplained.

Firstly, the flow of the entire processing performed in the informationprocessing system 1 is explained with reference to FIG. 25.

Here, in each of the Edges 50, the client device 2, the first serverapparatus 3 a, the first storage apparatus 10 a perform informationprocessing every day, and files are newly created and updated in thefirst storage apparatus 10 a every day along with the performance ofinformation processing.

As explained above, as for the files newly created in the first storageapparatus 10 a and the updated files, “1” is set in the metadatasynchronization necessity flag or the entity synchronization necessityflag in the inode management table 2212.

At a specified point of time everyday for example, the first storagesystem 8 a refers to the metadata synchronization necessity flag and theentity synchronization necessity flag in the inode management table2212, and selects the migration target files (S10000). Subsequently, thefirst storage system 8 a writes the information which specifies thetotal data amount of the files and the time period for migration(Window) and other information in the file migration information, andregisters the information in the file migration management table 110stored in the shared LU 173 in the second storage system 8 b (S10010).

Subsequently, when detecting that the file migration information isregistered in the file migration management table 110 (S10020), thesecond storage system 8 b determines whether migration can be performedor not based on the registered file migration information, theprocessing ability and the load at the point of time on the secondstorage apparatus 10 b (S10030). Subsequently, the second storage system8 b notifies the determination result to the first storage system 8 a(S10040).

The first storage system 8 a performs, is on standby for, or suspendsthe migration in accordance with the above-mentioned determinationresult (S10050).

When receiving the determination from the second storage system 8 b thatthe migration can be performed, the first storage system 8 a firstlyreplicates the migration target file stored in the first storageapparatus 10 a to the second storage apparatus 10 b, and performsmanagement by the management method by replication. The first storagesystem 8 a performs migration by continuing to stub the file.

The individual processing performed in the information processing system1 is explained below.

<Replication Processing>

Firstly, the processing of replicating files from the first storagesystem 8 a to the second storage system 8 b (hereinafter also referredto as the replication processing S2700) is explained with reference toFIG. 26 and FIG. 27.

The first server apparatus 3 a transfers the replication of thereplication target file stored in the first storage apparatus 10 a tothe second storage apparatus 10 b, and performs management by themanagement method by replication for the replication target file.

The management method by replication is a method by which data (metadataand entity) of a file is managed both in the first storage apparatus 10a and in the second storage apparatus 10 b. By the management method byreplication, when the entity or metadata of a file stored in the firststorage apparatus 10 a is updated, the metadata or entity of the file inthe second storage apparatus 10 b, which is managed as the replication(or the archive file) of this file is updated synchronously orasynchronously. By the management method by replication being performed,the consistency between the data (the metadata or the entity) of a filestored in the first storage apparatus 10 a and the data (the metadata orthe entity) of the file which is stored as the replication of the filein the second storage apparatus 10 b is secured (guaranteed)synchronously or asynchronously.

It should be noted that the metadata of a file (an archive file) in thesecond storage apparatus 10 b may also be managed as a file (as theentity of the file). By the above-mentioned method, the operation by themanagement method by replication can be performed even when thespecification of the file system 312 in the first server apparatus 3 aand the specification of the file system 342 in the second serverapparatus 3 b are different.

After registering the file migration information in the file migrationmanagement table 110, the first server apparatus 33 a monitors theresult of the determination as to whether migration can be performed ornot (S3811). After receiving the determination result that migration canbe performed (S3811: YES), among the files stored in the file storagearea in the first storage apparatus 10 a, the first server apparatus 3 aselects a file of which at least either the metadata synchronizationnecessity flag 2612 or the entity synchronization necessity flag 2613 isset to “1” as a migration target file (S3712, S3812).

Subsequently, the first server apparatus 3 a transmits the metadata orthe entity of the acquired file to the second server apparatus 3 b, andalso sets the metadata synchronization necessity flag 2612 of therelevant file to “0”, the entity synchronization necessity flag 2613 to“0”, and the replication flag 2614 to “1” (S3713, S3814).

After receiving the metadata or the entity (S3713, S3821), the secondserver apparatus 3 b stores the received metadata or entity in thesecond storage apparatus 10 b (S3714, S3822).

It should be noted that only the updated difference since the lastsynchronization may be transmitted instead of transmitting the entiremetadata or entity to the second server apparatus 3 b from the firstserver apparatus 3 a.

Replication is performed by the replication processing S2700, and thereplication of the data (the metadata and the entity) of the file storedin the first storage apparatus 10 a can be stored in the second storageapparatus 10 b.

<Stubbing Candidate Selection Processing>

Next, the processing performed in the information processing system 1for setting a replication file (a file managed by the replicationmanagement method (a file whose replication flag 2614 is set to “1”))stored in the first storage apparatus 10 a as a candidate for stubbing(hereinafter also referred to as stubbing candidate selection processingS2900) is described with reference to FIG. 28 and FIG. 29.

The first server apparatus 3 a selects a migration target file stored inthe first storage apparatus 10 a as the candidate for stubbing (S2911,S3013).

Next, after selecting the candidate for stubbing, the first serverapparatus 3 a sets the stubbing flag 2611 of the selected file to “1”,the replication flag 2614 to “0”, and the metadata synchronizationnecessity flag 2612 to “1” respectively (S2912, S3014).

<Stubbing Processing>

Next, the processing performed in the information processing system 1for actually stubbing the migration target file (hereinafter alsoreferred to as stubbing processing S3100) is shown in FIG. 30.Furthermore, the flowchart for explaining the details of the stubbingprocessing S3100 is shown in FIG. 31.

The stubbing processing S3100 is performed, for example, at the timingwhich is set in advance (for example, subsequent to the stubbingcandidate selection processing S2900), but the timing for starting thestubbing processing S3100 is not necessarily limited to the above.Hereinafter, the stubbing processing S3100 is explained with referenceto these figures.

The first server apparatus 3 a extracts one or more files as candidatesfor stubbing from the files stored in the file storage area in the firststorage apparatus 10 a (S3111, S3211, S3212).

Then, the first server apparatus 3 a deletes the entity of the extractedfile from the first storage apparatus 10 a (S3213), also sets an invalidvalue for the information indicating the storage destination of the filein the first storage apparatus 10 a from the metadata of the extractedfile (for example, setting a NULL value or zero in the field for settingthe storage destination of the relevant file of the metadata (forexample, the setting field of the block address 2618) (S3214)), andactually stubs the stubbing candidate file (S3112). Furthermore, at thispoint, the first server apparatus 3 a sets the metadata synchronizationnecessity flag 2612 to “1” (S3215).

<Metadata Access Processing>

Next, the processing performed in the information processing system 1 ina case the file system 312 of the first server apparatus 3 a receives anaccess request (a reference request or an update request) for themetadata of a stubbed file (a file whose stubbing flag 2611 is set to“1”, which is also referred to as a stubbed file) from the client device2 and the like (hereinafter also referred to as metadata accessprocessing S3900) is shown in FIG. 32. Furthermore, a flowchart forexplaining the details of the metadata access processing S3900 is shownin FIG. 33. The metadata access processing S3900 is explained withreference to these figures.

The first server apparatus 3 a monitors whether or not an access request(a reference request or an update request) for the metadata of a stubbedfile is received from the client device 2 (S4011). When receiving theaccess request for the metadata of a stubbed file (S3911, S4011: YES),the first server apparatus 3 a acquires the metadata of the firststorage apparatus 10 a which is the target of the received accessrequest (S4012) and, in accordance with the received access request(S4013), refers to the metadata (transmission of response informationbased on the read metadata to the client device 2) (S4014) or the updateof the metadata (S3912, S4015). It should be noted that the first serverapparatus 3 a sets the metadata synchronization necessity flag 2612 ofthe file to “1” (S3913) when the first server apparatus 3 a has updatedthe contents of the metadata (S4015).

As explained above, when an access request for a stubbed file occurs andwhen the target of the access request is only the metadata of the file,the first server apparatus 3 a processes the access request by using themetadata stored in the first storage apparatus 10 a. Therefore, when thetarget of the access request is only the metadata of the file, the firstserver apparatus 3 a can respond to the client device 2 immediately.

<Stubbed File Entity Reference Processing>

Next, processing performed in the information processing system 1 in acase the first server apparatus 3 a receives a reference request for theentity of a stubbed file from the client device 2 (hereinafter alsoreferred to as stubbed file entity reference processing S4100) is shownin FIG. 34. Furthermore, a flowchart for explaining the details of thestubbed file entity reference processing S4100 is shown in FIG. 35.Hereinafter, the stubbed file entity reference processing S4100 isexplained with reference to these figures.

When receiving a reference request for the entity of a stubbed file fromthe client device 2 (S4111, S4211: YES), the first server apparatus 3 adetermines whether or not the entity of the relevant stubbed file isstored in the first storage apparatus 10 a (S4112, S4212). Thisdetermination is performed, for example, based on whether or not a validvalue is set for the information (for example, the block address 2618)indicating the storage destination of the entity of the stubbed file inthe acquired metadata.

When the entity of the stubbed file is stored in the first storageapparatus 10 a (S4212: YES), the first server apparatus 3 a reads theentity of the relevant stubbed file from the first storage apparatus 10a, generates information to respond to the client device 2 based on theread entity, and transmits the generated response information to theclient device 2 (S4113, S4213).

Meanwhile, when the entity of the stubbed file is not stored in thefirst storage apparatus 10 a (S4212: NO), the first server apparatus 3 arequests for the entity of the stubbed file to the second serverapparatus 3 b (hereinafter also referred to as a recall request) (S4114,S4214). It should be noted that the acquisition request for the entityneed not be a single request acquiring the entire entity and, forexample, only a part of the entity may be requested a plurality oftimes.

When receiving the entity of the stubbed file transmitted in response tothe above-mentioned acquisition request from the second server apparatus3 b (S4221, S4222, S4215, S4115), the first server apparatus 3 agenerates response information based on the received entity andtransmits the generated response information to the client device 2(S4116, S4216).

Furthermore, the first server apparatus 3 a stores in the first storageapparatus 10 a the entity received from the above-mentioned secondserver apparatus 3 b, and sets the contents indicating the storagedestination in the first storage apparatus 10 a of the relevant file forthe information indicating the storage destination of the entity of therelevant file of the metadata of the relevant stubbed file (for example,the block address 2618) (S4217).

Furthermore, the first server apparatus 3 a sets the stubbing flag 2611of the relevant file to “0”, the replication flag 2614 to “0”, and themetadata synchronization necessity flag 2612 to “1” respectively (S4117,S4218).

It should be noted that “1” is set for the metadata synchronizationnecessity flag 2612 for automatically synchronizing, in an ex post factomanner, the contents of the stubbing flag 2611 and the replication flag2614 of the relevant stubbed file between the first storage apparatus 10a and the second storage apparatus 10 b.

<Stubbed File Entity Update Processing>

Next, processing performed in the information processing system 1 in acase the first server apparatus 3 a accepts an update request for theentity of a stubbed file from the client device 2 (hereinafter alsoreferred to as stubbed file entity update processing S4300) is shown inFIG. 36. Furthermore, the flowchart for explaining the details of thestubbed file entity update processing S4300 is shown in FIG. 37.Hereinafter, the stubbed file entity update processing S4300 isexplained with reference to these figures.

When receiving an update request for the entity of a stubbed file fromthe client device 2 (S4311, S4411: YES), the first server apparatus 3 adetermines whether or not the entity of the relevant stubbed file isstored in the first storage apparatus 10 a (S4312, S4412). It should benoted that the determination method is the same as in the case of thestubbed file entity reference processing S4100.

When the entity of the stubbed file is stored in the first storageapparatus 10 a (S4412: YES), the first server apparatus 3 a updates theentity of the relevant stubbed file stored in the first storageapparatus 10 a in accordance with the contents of the update request(S4413), and also sets the entity synchronization necessity flag 2613 ofthe relevant stubbed file to “1” (S4313, S4414).

Meanwhile, as a result of the above-mentioned determination, when theentity of the stubbed file is not stored in the first storage apparatus10 a (S4412: NO), the first server apparatus 3 a transmits anacquisition request for the entity of the relevant stubbed file (arecall request) to the second server apparatus 3 b (S4314, S4415).

When receiving the entity of the file transmitted from the second serverapparatus 3 b in accordance with the above-mentioned request (S4315,S4421, S4422, S4416), the first server apparatus 3 a updates thecontents of the received entity in accordance with the contents of theupdate request (S4417), and stores the updated entity as the entity ofthe relevant stubbed file in the first storage apparatus 10 a (S4316,S4418).

Furthermore, the first server apparatus 3 a sets the stubbing flag 2611of the relevant stubbed file to “0”, the replication flag 2614 to “0”,the metadata synchronization necessity flag 2612 to “1”, and the entitysynchronization necessity flag to “1” respectively (S4419).

<Migration Determination Processing>

Next, the migration determination processing (S10030 in FIG. 25)performed by the second server apparatus 3 b is explained with referenceto FIG. 38. The migration determination processing is processing fordetermining whether migration can be performed or not when the secondserver apparatus 3 b detects that file migration information is newlyregistered in the file migration management table 110, based on theregistered file migration information and the load and the processingability of the second storage apparatus 10 b at that point of time.

Firstly, the second server apparatus 3 b calculates the processible dataamount of the second storage apparatus 10 b at the current point of time(S10110). The processible data amount of the second storage apparatus 10b can be ascertained, for example, from the difference between the upperlimit value (processing ability) for example, of the processible dataamount which is set for the second storage apparatus 10 b in advance,and the current load on the second storage apparatus 10 b (the load dueto migration which is currently under process). The current load can beacquired from the load acquisition unit 347.

For example, when the processing ability of the second storage apparatus10 b (for example, the upper limit value of the processible data amount)is five gigabytes per second and when one gigabyte of data per second iscurrently being migrated from another first storage system 8 a, theprocessible data amount can be calculated as four gigabytes per second.

Next, the second server apparatus 3 b refers to the file migrationmanagement table 110, and compares the load on the second storageapparatus 10 b (the increment of the load) which increases when themigration registered in the file migration management table 110 asmentioned above is performed with the calculated processible data amountmentioned above (S10120).

When the load on the second storage apparatus 10 b which increases whenthe migration is performed does not exceed the processible data amount,the process proceeds to No and, corresponding to the above-mentionedregistered file migration information, enters “ALLOWED” indicating thatthe migration can be performed (S10130).

Meanwhile, when the load on the second storage apparatus 10 b whichincreases when the above-mentioned migration is performed exceeds theprocessible data amount, the process proceeds to Yes. Subsequently, thesecond server apparatus 3 b refers to the “WINDOW” field in the filemigration information and confirms by what time the migration must becompleted (time limit for migration).

Furthermore, the second server apparatus 3 b estimates the time requiredfor the migration based on the total data amount written in the “TOTALTRANSFER AMOUNT” field and the transfer rate information written in the“EFFECTIVE PERFORMANCE” field in the file migration information.Subsequently, based on the current time and this required time, thesecond server apparatus 3 b ascertains the end time of the migration(migration completion time). Subsequently, the second server apparatus 3b determines whether the migration can be completed by the time limitfor migration or not (S10140).

As a result, if the migration cannot be completed by the above-mentionedtime limit for migration, the process proceeds to Yes and, correspondingto the above-mentioned registered file migration information, enters“NOT ALLOWED” indicating that migration cannot be performed (S10150).

Meanwhile, if the migration can be completed by the above-mentioned timelimit for migration, the process proceeds to No and, corresponding tothe above-mentioned registered file migration information, enters“STANDBY” indicating that the migration is to standby temporarily(S10160).

As explained above, the second server apparatus 3 b enters the result ofthe determination as to whether migration can be performed or not in the“RESULT” field of the file migration management table 110. The contentsof the table are shown in FIG. 19.

<Re-Evaluation Processing>

Subsequently, the second server apparatus 3 b refers to the filemigration management table 110 again at a specified timing, and performsre-evaluation processing for performing the above-mentioneddetermination for the file migration information for which migration isnot performed. This re-evaluation processing is performed, for example,at a timing when the migration processing which has been alreadyperformed is completed, or after a specified period of time (forexample, an hour later), and the like. This re-evaluation processing isexplained with reference to the flowchart shown in FIG. 39.

Firstly, the second server apparatus 3 b refers to the file migrationmanagement table 110 and determines whether or not any unprocessedrecall processing is registered (S10210). The file migration informationin which R is entered in the “READ/WRITE” field of the file migrationmanagement table 110 indicates that the processing is the recallprocessing. For example, in the example shown in FIG. 19, file migrationinformation #6 is a recall processing.

When the recall processing is requested, the processing ispreferentially processed because the entity of the file must betransmitted to the request source before the first storage system 8 abeing the request source detects a timeout error.

If recall processing is registered in the file migration managementtable 110, the process proceeds to Yes and enters “ALLOWED” in the“RESULT” field of the file migration information indicating the recallprocessing (S10220). In this way, recall processing is performedimmediately.

Meanwhile, if no unprocessed recall processing is registered in the filemigration management table 110, the process proceeds to No and selectsfile migration information for which “STANDBY” is determined from thefile migration management table 110 (S10230). For example, the secondserver apparatus 3 b refers to the “REMAINING CACHE” field in the filemigration management table 110 and selects the file migrationinformation with the smallest remaining storage capacity from the filemigration information for which “STANDBY” is determined.

Subsequently, the second server apparatus 3 b calculates the currentprocessible data amount of the second storage apparatus 10 b (S10240).The second server apparatus 3 b compares the load on the second storageapparatus 10 b which increases when the migration by the selected filemigration information mentioned above is performed with the calculatedprocessible data amount mentioned above (S10250).

If the load on the second storage apparatus 10 b which increases whenthe migration is performed does not exceed the processible data amount,the process proceeds to No and, corresponding to the above-mentionedselected file migration information, enters “ALLOWED” indicating thatthe migration can be performed (S10260).

If the load on the second storage apparatus 10 b which increases whenthe migration is performed exceeds the processible data amount, theprocess proceeds to Yes, selects the next candidate from the other filemigration information which is registered in the file migrationmanagement table 110 and for which “STANDBY” is determined (S10270).

The next candidate can be, for example, a file migration informationwhose remaining storage capacity is the second smallest. Or, themigration completion time for each migration may be ascertained based oneach file migration information and set as the next candidate the filemigration information whose time length from migration completion timeuntil the time limit for migration written in the “WINDOW” field of thefile migration management table 110, is the shortest.

Since it can be considered that the first storage apparatus 10 a with asmall remaining storage capacity is more likely to run out of storagearea first, by the first storage apparatus 10 a with a smaller remainingstorage capacity being set to be preferentially selected as thecandidate for migration, migration of the file stored in the firststorage apparatus 10 a which is more likely to run out of storage areacan be preferentially performed. Therefore, the reliability of theinformation processing system 1 can be further improved.

Or, by preferentially setting as the candidate for migration the filemigration information with a shorter time length from migrationcompletion time until the time limit for migration, the number ofmigrations suspended can be reduced. Therefore, more files can bemigrated to the second storage apparatus 10 b in the informationprocessing system 1, so that the reliability of the informationprocessing system 1 can be further improved.

It should be noted that in the flowchart shown in FIG. 39 recallprocessing is performed at steps S10210 and S10220 after confirmingwhether any unprocessed recall processing is registered or not. However,processing at steps S10210 and S10220 becomes unnecessary, for example,when a communication band for the recall processing between the firststorage system 8 a and the second storage system 8 b is secured andrecall processing is performed preferentially by using thiscommunication band for recall processing. In this way, recall processingcan be performed without delay.

<Processing in Case Migration Cannot be Performed>

Next, processing in a case it is determined, as a result of themigration determination processing (S10030 in FIG. 25), that migrationcannot be performed is explained with reference to FIGS. 40 to 42.

FIG. 40 is a diagram showing the configuration for performing processingin a case migration cannot be performed. In this case, the informationprocessing system 1 is communicatively coupled to the third storagesystem 8 c via the second network 7.

The third storage system 8 c may be, for example, a storage system or astorage apparatus which is installed at a data center of anotherinformation processing system different from the information processingsystem 1, and may also be a storage system or a storage apparatusstructured in another Core if a plurality of Cores 51 are set ininformation processing system 1.

An LU 173 exclusively assigned to each of the plurality of Edges 50 inthe information processing system 1 and a shared LU 173 shared by allthe Edges 50 are also configured in the third storage system 8 c, and acoordinator server 346, a data replication/migration processing unit344, and the like are also structured. And, the third storage system 8 cperforms the same information processing as the second storage system 8b. Furthermore, the third storage system 8 c is configured to be capableof performing information processing in cooperation with the secondstorage system 8 b.

Therefore, the first storage system 8 a in the Edge 50 in theinformation processing system 1 is configured to be accessible to thesecond storage system 8 b and the third storage system 8 c.

As shown in the flowchart in FIG. 41, when the second storage system 8 bnotifies the first storage system 8 a of the determination thatmigration cannot be performed (S20310), the second storage system 8 brequests the third storage system 8 c to determine whether or notmigration from the first storage system 8 a can be performed or not(S20320). This request also includes file migration information from thefirst storage system 8 a for which determination was made that migrationis not allowed.

Furthermore, when acquiring the determination result that migration bythe second storage system 8 b cannot be performed, the first storagesystem 8 a waits to be transmitted a determination result as to whetheror not migration to the third storage system 8 c can be performed.

The third storage system 8 c determines whether or not migration fromthe first storage system 8 a can be performed based on the filemigration information transmitted from the second storage system 8 b,and the processing ability of the third storage system 8 c and the loadat the point of time on the third storage system 8 c (S20330). Themethod of this determination processing can be the same as the method ofthe determination processing in the above-mentioned second storagesystem 8 b.

The third storage system 8 c transmits the result of the determinationto the second storage system 8 b (S20340). If the migration cannot beperformed (S20350), the third storage system 8 c terminates theprocessing (S20360).

When receiving the above-mentioned determination result from the thirdstorage system 8 c, the second storage system 8 b transmits thedetermination result to the first storage system 8 a (S20370).

When the determination result is “ALLOWED” (S20380), the first storagesystem 8 a transfers the migration target file to the third storagesystem 8 c (S20400). Subsequently, the third storage system 8 c receivesthe migration target file, and writes the file to the LU 173 assigned tothe first storage system 8 a (S20410).

It should be noted that the second storage system 8 b may be made toacquire the file which the third storage system 8 c received from thefirst storage system 8 a from the third storage system 8 c at such as atiming when the load falls below a specified value, and store the filein the LU 173 of the second storage system 8 b.

Meanwhile, when the determination result is “STANDBY” or “NOT ALLOWED”at S20380, the first storage system 8 a is at a standby for or suspendsthe migration (S20390).

By controlling processing in the above manner, even if the migration tothe second storage system 8 b cannot be performed, the first storagesystem 8 a can perform the migration to the third storage system 8 c. Inthis way, it becomes possible to reduce the number of migrations to besuspended. Therefore, since more files can be migrated in theinformation processing system 1, the reliability of the informationprocessing system 1 can be further improved.

Furthermore, the processing in the case migration is suspended at theabove-mentioned S20390 is explained with reference to FIG. 42.Specifically speaking, the processing for the case neither the migrationto the second storage system 8 b nor the migration to the third storagesystem 8 c could be performed.

When file migration cannot be performed, the first storage system 8 acannot stub the file, and thus straining the storage capacity of thefirst storage system 8 a.

Therefore, in this case, the first storage system 8 a monitors theremaining storage capacity of the first storage apparatus 10 a andinhibits data write from the client device 2 to the storage apparatus 10a when it is detected that the storage capacity falls below a specifiedvalue. Or, the first storage system 8 a monitors the usage rate of thestorage capacity of the first storage apparatus 10 a and inhibits datawrite from the client device 2 to the storage apparatus 10 a when it isdetected that the usage rate exceeds a specified value (for example,90%).

In this case, the first storage system 8 a returns an error to theclient device 2 when a data write request is made from the client device2 to the first storage apparatus 10 a.

It should be noted that, if the first storage system 8 a cannot receivethe result of the determination which is “yes” for migration even afterthe elapse of a predetermined amount of time after transmitting the filemigration information to the second storage system 8 b, the firststorage system 8 a can control the processing at that point of time sothat data write from the client device 2 to the first storage apparatus10 a is inhibited.

In this way, for example, the overflow of the storage area of the firststorage apparatus 10 a can be prevented before the second storage system8 b determines that migration cannot be performed in case the usedamount of the storage area of the first storage apparatus 10 a increasesrapidly and the like.

Or, the first storage system 8 a can control the processing so that datawrite from the client device 2 to the first storage apparatus 10 a isinhibited at the point of time when the first storage system 8 areceives the result of the determination from the second storage system8 b that migration cannot be performed.

As explained above, by suspending write to the first storage apparatus10 a at an earlier stage, the prevention of the overflow of the storagearea of the first storage apparatus 10 a can be further ensured.

Furthermore, the first storage system 8 a may also accumulate theresults of determinations performed by the second storage system 8 b asto whether migration can be performed or not and perform analysisprocessing for the results on the determinations. For example, when therate of determinations that migration cannot be performed is higher thana specified value, the first storage system 8 a may notify an alertmessage to the system administrator.

Or, the first storage system 8 a can classify the results of thedeterminations as to whether migration can be performed or not, forexample, in more detail, into days of the week, months, data size ofmigration, and the like and notify the result of the analysis to thesystem administrator.

In this way, information used for reviewing and determining the contentsof the file migration information policy can be provided to the systemadministrator.

Although the embodiments of the present invention have been explainedabove, the embodiments above are intended for facilitating theunderstanding the present invention and not for limiting theinterpretation of the present invention to the embodiments above. Thepresent invention can be modified or improved without departing form thegist thereof, and equivalents thereof are also encompassed by thepresent invention.

The invention claimed is:
 1. An information processing systemcomprising: a plurality of first storage apparatuses configured toreceive data I/O requests in file units sent from an informationprocessing apparatus, and perform data I/O; and a second storageapparatus that is communicatively coupled to each of the plurality offirst storage apparatuses, wherein: one of the plurality of firststorage apparatuses, prior to a file migration to the second storageapparatus, is configured to notify the second storage apparatus of filemigration information being information relating to the file migration,the second storage apparatus is configured to calculate an increment ofa load on the second storage apparatus that is generated by the filemigration based on information written in the file migrationinformation, the second storage apparatus is configured to determinewhether the file migration is allowable based on a current load on thesecond storage apparatus itself and the increment, the second storageapparatus is configured to notify the determination result to the one ofthe plurality of first storage apparatuses that has notified the filemigration information, the one of the plurality of first storageapparatuses is configured to determine whether to migrate the file tothe second storage apparatus based on the determination result, the oneof the plurality of first storage apparatuses is configured to send areplica of the file to the second storage apparatus, acquire from thesecond storage apparatus storage location of the replica of the file inthe second storage apparatus, and store the acquired storage location inthe one of the plurality of first storage apparatuses and delete thefile from the one of the plurality of first storage apparatuses, thefile migration information includes information indicating a usage rateof the one of the plurality of first storage apparatuses, and the secondstorage apparatus is further configured to refer to the informationindicating the usage rate in the file migration information notifiedfrom the one of the plurality of first storage apparatuses to determinewhether to proceed with the file migration or not, wherein theinformation indicating the usage rate includes information indicating aremaining storage capacity of the one of the plurality of first storageapparatuses.
 2. The information processing system according to claim 1,wherein: the second storage apparatus includes a file migrationmanagement table capable of being read and written from the plurality offirst storage apparatuses, the one of the plurality of first storageapparatuses, prior to migration of a file to the second storageapparatus is configured to notify the second storage apparatus of thefile migration information by writing the file migration information inthe file migration management table, and the second storage apparatus isconfigured to notify the one of the plurality of first storageapparatuses of the determination result by writing the determinationresult in the file migration management table in correspondence with thefile migration information.
 3. The information processing systemaccording to claim 1, wherein the second storage apparatus is configuredto determine that the file migration from the one of the plurality offirst storage apparatuses is allowable, in a case a sum of the incrementand current load on the second storage apparatus is equal to or lessthan a predetermined value.
 4. The information processing systemaccording to claim 1, wherein the second storage apparatus is configuredto determine that the file migration from the one of the plurality offirst storage apparatuses is to be at hold, in a case a sum of theincrement and a current load on the second storage apparatus is greaterthan a predetermined value.
 5. The information processing systemaccording to claim 4, wherein the second storage apparatus, at apredetermined timing, is configured to redetermine based on the filemigration information whether the file migration determined to be athold is allowable.
 6. The information processing system according toclaim 5, wherein the second storage apparatus, when redeterminingwhether to perform the file migration, is configured to refer toinformation indicating the usage rate included in each file migrationinformation, and redetermine, in descending order of the usage rate,file migration information notified from the first storage apparatuses,when file migration is determined to be at hold for another of the firststorage apparatuses.
 7. The information processing system according toclaim 5, wherein the file migration information further includesinformation indicating a data amount of a file to be migrated, timelimit for migration, and data transfer rate at migration, and the secondstorage apparatus, when redetermining whether to perform the filemigration, is configured to ascertain transfer completion time for eachfile, based on data amount, data transfer rate at migration and currenttime written in each file migration information when file migration isdetermined to be at hold for another of the first storage apparatuses,and the second storage apparatus is configured to redetermine filemigration information in ascending order of length of time frommigration completion time until migration time limit.
 8. The informationprocessing system according to claim 7, wherein the second storageapparatus is configured to determine that the file migration from theone of the plurality of first storage apparatuses is not allowed whenthe file migration cannot be completed by the migration time limit. 9.The information processing system according to claim 8, wherein each ofthe plurality of first storage apparatuses are communicatively coupledto a third storage apparatus, and the one of the plurality of firststorage apparatuses that was determined that the file migration is notallowed, is configured to migrate the file to the third storageapparatus.
 10. The information processing system according to claim 8,wherein: the one of the plurality of first storage apparatuses that wasdetermined that the file migration is not allowed, prevents theinformation processing apparatus from writing a file in the one of theplurality of first storage apparatuses, and the second storage apparatusis configured to acquire the file stored in the third storage apparatusat a prescribed time.
 11. The information processing system according toclaim 1, wherein: the one of the plurality of first storage apparatusesis further configured to perform a recall processing to transmit thereplica of the file from the second storage apparatus to the one of theplurality of first storage apparatuses in response to a file beingaccessed by the information processing apparatus being a stub file; andthe one of the plurality of first storage apparatuses is furtherconfigured to perform the recall process in preference to a filemigration process.
 12. The information processing system according toclaim 11, wherein the stub file comprises the acquired storage locationfrom the second storage apparatus.
 13. A method for controlling aninformation processing system including a plurality of first storageapparatuses configured to receive data I/O requests in file units sentfrom an information processing apparatus, and perform data I/O, and asecond storage apparatus that is communicatively coupled to each of theplurality of first storage apparatuses, the method comprising:notifying, by one of the plurality of first storage apparatuses, priorto file migration to the second storage apparatus, to the second storageapparatus of file migration information being information relating tothe file migration and including information indicating a usage rate inthe one of the plurality of first storage apparatuses; calculating, bythe second storage apparatus, an increment of a load on the secondstorage apparatus that is generated by the file migration based oninformation written in the file migration information; determining, bythe second storage apparatus, whether the file migration is allowablebased on a current load on the second storage apparatus itself and theincrement, notifying, by the second storage apparatus, the determinationresult to the one of the plurality of first storage apparatuses that hasnotified the file migration information; determining, by the one of theplurality of first storage apparatuses, whether to migrate the file tothe second storage apparatus based on the determination result; sending,by the one of the plurality of first storage apparatuses, a replica ofthe file to the second storage apparatus; acquiring, by the one of theplurality of first storage apparatuses from the second storageapparatus, storage location of the replica of the file in the secondstorage apparatus; storing, by the one of the plurality of first storageapparatuses, the acquired storage location in the one of the pluralityof first storage apparatuses and deleting the file from the one of theplurality of first storage apparatuses; and referring, by the secondstorage apparatus, to the information indicating the usage rate in thefile migration information notified from the one of the plurality offirst storage apparatuses to determine whether to proceed with the filemigration or not, wherein the information indicating the usage rateincludes information indicating a remaining storage capacity of the oneof the plurality of first storage apparatuses.
 14. The method accordingto claim 13, further comprising: performing, by the one of the pluralityof first storage apparatuses, a recall processing to transmit thereplica of the file from the second storage apparatus to the one of theplurality of first storage apparatuses in response to a file beingaccessed by the information processing apparatus being a stub file, andwherein the recall process is performed in preference to a filemigration process.
 15. The method according to claim 14, wherein thestub file comprises the acquired storage location from the secondstorage apparatus.